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Appendix A. Methods 


Reliability of teacher and student surveys 


Instructional leaders in the three districts (Chandler Unified School District, Flagstaff Unified School District, and 
Sunnyside Unified School District) derived the study’s survey items from existing survey instruments in summer 
2018. The items examining teachers’ core formative assessment practices were drawn from the relevant subscales 
on the Formative Assessment Rubrics, Reflection and Observation Protocol (Wylie & Lyon, 2016). The items for 
students were drawn from the self-regulated learning subscales on the Motivated Strategies for Learning 
Questionnaire (Pintrich & DeGroot, 1990) and the Organisation for Economic Co-operation and Development 
Programme for International Student Assessment (PISA; Artelt et al., 2003). The reliability (Cronbach’s alpha) for 
the Motivated Strategies for Learning Questionnaire self-regulation subscale was .75 (Ilker et al., 2014), while the 
reliability of the relevant PISA learning strategies subscale, in its U.S. administration, was .83 (Artelt et al., 2003). 


In the current study the scale reliability coefficient (Cronbach’s alpha) was .79 for the seven student survey items 
and .77 for the six teacher survey items. 


Frequency calculations and proportional weighting 

The initial analyses for the study tabulated how often (from 1-Never to 5-Always) the responding teachers and 
students reported using these different formative assessment practices and self-regulated learning strategies in 
the classroom in an average or normal week. Next, the study team calculated a frequency index for each teacher 
and student that represented the sum of each respondent’s 1-5 ratings across all questions and then further 
explored (through correlations) the association between these frequencies among surveyed teachers and their 
surveyed students. 


A higher proportion of secondary school students than of elementary school students responded to the survey 
(table A1). In response, the study team adjusted (using proportional weighting) the student survey data so that 
the responding student sample at every grade level accurately reflected that grade level’s proportional 
representation in each district’s student population. For each student survey question, a side-by-side comparison 
of the raw, unweighted response frequency and its weighted counterpart showed that the weighting had very 
little impact on the results (table A2). The differences between the mean responses for the weighted and 
unweighted groups varied by less than .05 standard deviation units for each survey question. 
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Table A1. Proportions of students among survey respondents and among the study population in the three 
Arizona study districts, by grade span, 2019 (percent) 


Respondents Study population 
Grade span (eet f0)) (n = 49,782) 
Elementary (grades 3-5) 21.1 29.4 
Secondary (grades 6-10) 78.9 70.6 


Source: Chandler Unified School District, Flagstaff Unified School District, and Sunnyside Unified School District data for 2019. 


Table A2. Comparison of unweighted and weighted frequencies of self-regulated learning strategies used by 
students in the three Arizona study districts during a normal week, 2019 (percent) 


About half the time Most of the time 


Self-regulated 

learning strategy Unweighted Weighted Unweighted Weighted Unweighted Weighted Unweighted Weighted Unweighted Weighted 
S1. | set goals for 4.9 4.8 16.5 16.3 26.9 26.9 36.6 36.6 15.2 15.4 
myself to guide 

my learning in 

class. 


S2. In class | ask 7.0 7.1 26.5 26.4 29.5 29.4 27.0 26.9 10.0 10.1 
for feedback from 

the teacher to 

check my 

understanding. 

S3. | try to 3.8 3.8 12.6 12.5 24.5 24.3 37.1 37.1 21.9 22.2 
connect what I’m 

learning to things 

| already know. 

S4. In class | ask 7.7 7.7 19.3 19.2 27.5 27.2 28.9 28.8 16.6 17.1 
myself questions 

to help me 

understand what 

I’m learning. 

S5. In class | get 10.2 10.6 24.7 25.2 26.4 26.4 25.9 25.3 12.8 12.6 
feedback from 

other students to 

improve my work. 

S6. In class | keep 3.6 3.6 9.7 9.9 19.2 19.4 32.6 32.1 34.9 34.9 
track of my own 

progress. 

S7. In class | 3.3 3.3 11.6 11.6 26.5 26.1 35.4 35.3 23.1 23.8 
identify different 

ways to improve 

my work. 


Source: Chandler Unified School District, Flagstaff Unified School District, and Sunnyside Unified School District data for 2019. 
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Appendix B. Supporting analyses 

The rank-order correlations between each teacher survey item (formative assessment practice) and each student 
survey item (self-regulated learning strategy) are listed in table B1. The rank-order correlations provide insight 
into how the frequency of each formative assessment practice is associated with the frequency of each self- 
regulated learning strategy. As shown, teachers providing structured occasions for students to provide feedback 
to one another and assess their own learning had small, statistically significant positive associations with the 
(average) frequency of each self-regulated learning strategy that their students reported. 


Table B1. Students in the three Arizona study districts used self-regulated learning strategies more frequently 
when their teachers facilitated student peer feedback and self-assessment more frequently, 2019 
Association between the average frequency of the teachers’ formative assessment practice and the average frequency of 
the self-regulated learning strategy applied by their students each week 
Student self-regulated learning strategy 
$3. I try to S4. In class | 


S1. | set S2. In class | connect ask myself S5. In class | S7. In class | 

goals for ask for what I’m questionsto get feedback identify 

myself to feedback from __ learning to help me from other S6. In class | different 

guide my the teacher to things | Wlate(=1acim-l ale] students to keep track of VEN Ame) 
Teacher formative learning in check my already what I’m improve my my own improve my 
assessment practice class. Wiavel=yasim-Vatel (aloe know. learning. Niela.e progress. Nelda 
T1. At some point in -.03 —.04 05 —.04 —.01 .09** —.00 
the lesson, | 


communicate the 
learning goal and 
success criteria (i.e., 
what quality work 
looks like) for the 
lesson to my students. 
T2. | help my students .06 05 05 05 .06 .09** .06* 
understand what 
meeting the goal and 
criteria means for the 
lesson. 
T3. | provide feedback 05 .06 05 .08* .03 .07* 05 
to students that helps 
them take steps for 
improvement. 
T4. | model for .09** .06 05 .06 13** .08* 0** 
students how to give 
constructive feedback 
to their peers. 
T5. | provide .08* .08* .09** .09** 16** Ai .09** 
structured occasions 
for students to 
provide feedback to 
one another. 
T6. My students .09** .06 .09** .08* 11** 14** .08* 
assess their own 
learning and think 
about next steps in 
class. 
* Significant at p < .05; ** significant at p < .01; *** significant at p < .001. 
Note: n = 998 teacher respondents whose students also completed a survey. Values in the table reflect the strength and consistency (monotonicity) of the 
association, as indicated by Spearman (nonparametric) rank-order correlation coefficients (9). 
Source: Chandler Unified School District, Flagstaff Unified School District, and Sunnyside Unified School District data for 2019. 
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Tables B2 and B3 present additional details about the differences between teachers trained and untrained in 
formative assessment practices and the self-regulated learning strategies that their students reported using ina 
normal week (see table 6 in the main report). 


Table B2. Students in the three Arizona study districts in classrooms with teachers who participated in any 
formative assessment coursework reported using self-regulated learning strategies more frequently, 2019 


With formative assessment Without formative assessment 
coursework coursework Sclaciue 


Pooled 
Teacher/classroom Number of Number of INIUTaa) oX=1 ane) me Neal o>) are) Sielalerlael 
type Mean teachers students Mean teachers students Difference Effectsize deviation 


Overall 3.58 367 3,316 3.44 610 6,695 0.14** 0.23 0.64 
Elementary 3.71 135 1,373 3.61 192 3,348 0.09 0.14 0.68 
Secondary 3.53 208 1,654 3.37 363 2,595 0.16** 0.26 0.61 
STEM 3.52 104 624 3.43 180 1,509 0.09 0.16 0.57 
Non-STEM 3.61 263 2,692 3.44 430 5,186 0.16** 0.25 0.66 


* Significant at p < .05; ** significant at p < .01; *** significant at p < .001. 

STEM is science, technology, engineering, and math. 

Note: This table displays results for teacher respondents whose students also completed a survey. The teacher and student counts represent the number 
of unique people contributing to the group mean for each specific row group. Statistical significance was estimated using independent t-tests of the group 
means. 

Source: Chandler Unified School District, Flagstaff Unified School District, and Sunnyside Unified School District data for 2019. 


Table B3. Students in the three Arizona study districts in classrooms with teachers who participated in any 
type of formative assessment training reported using self-regulated learning strategies more frequently, 2019 


With any type of formative Without formative 
assessment training assessment training Statistic 
Pooled 
Teacher/classroom Number of Number of Number of Number of Sieelalerlael 
type Mean teachers students teachers students Difference Effect size deviation 
Overall 3.50 876 9,038 3.39 101 973 0.11 * 0.18 0.64 
Elementary 3.68 294 4,088 3.36 33 633 0.32 ** 0.48 0.68 
Secondary 3.43 514 4,029 3.39 57 220 0.05 0.08 0.61 
STEM 3.45 256 1,967 3.57 28 166 —0.12 0.24 0.57 
Non-STEM 3.53 620 7,071 3.32 73 807 0.21 ** 0.21 0.66 


* Significant at p < .05; ** significant at p < .01; *** significant at p < .001. 

STEM is science, technology, engineering, and math. 

Note: This table displays results for teacher respondents whose students also completed a survey. The teacher and student counts represent the number 
of unique people contributing to the group mean for each specific row group. Statistical significance was estimated using independent t-tests of the group 
means. 

Source: Chandler Unified School District, Flagstaff Unified School District, and Sunnyside Unified School District data for 2019. 
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